Generic Mechanism for Reducing Repetitions in Encoder-decoder Models

Encoder-decoder models have been commonly used; they achieved state-of-the-art results for many natural language generation tasks. However, according to the reports of previous studies, encoder-decoder suffer from generating redundant repetitions. Thus, we herein propose a repetition reduction module (RRM) that estimates semantic difference source sentence before and after it is fed into model capture consistency between two sides. As an autoencoder, proposed mechanism supervises training reduce number repeatedly generated tokens. The evaluation publicly available machine translation response datasets demonstrate effectiveness our proposal.